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vThe model on which this research is based proposes that an 
instructional strategy has a direct effect (on the objective it 
is directed toward) and indirect effects (on related objectives) 


Most training situations aim at teaching a group of related 
objectives. FbHa ving the reocmendaticn of current learning 
theory, a maximally effective training sequence would consist 
of a separate instructional strategy to teach each objective. 
This is costly in time. The model suggests that it might not 
be necessary to use a separate strategy to train each objective 
if indirect effects allow us to achieve more than one objective 
at a time. Accaiplishing more than one objective at a time 
results in more efficient training. 

Seven objectives associated with mastering vocabulary 
concepts were identified and an instructional strategy designed 
for each. Two of the strategies were tried-out. Early results 
show that sane methods produce high levels of achievement on 
more than one objective. The goal of our continuing research 
is to select the most efficient combination of strategies that 
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SUMMARY 


This program of research has set out to address the issue of 
efficiency in .instruction in teaching vocabulary concepts. There 
are a number - of performance outcomes associated with "knowing" a 
word; we have identified seven such outcomes and have developed 
seven instructional modules, one targeted toward each out care. 
Instruction of this sort should be effective, but not necessarily 
efficient, in the sense that a great deal of learning time would 
be spent going through each of the seven modules in order to learn 
each outcome. We hypothesize that it is not necessary for sub¬ 
jects to receive instruction via all seven modules in order to 
achieve acceptable levels of performance on the criterion tests 
measuring each outcome. We hypothesize (in a quantitative repre¬ 
sentation of the design problem) that each instructional module 
will have a direct effect, shewn by performance on its related 
criterion test, and that it will also have an indirect effect, 
smaller than its direct effect, cn the criterion tests measuring 
tiie other learning outcomes. If this is true, seme smaller combina¬ 
tion of teaching modules than the full set should produce acceptable 
levels of performance cn the outcome measures, at a lower cost. 

The purpose of cur program of research is to evaluate the hypothesis 
and to determine the optimal canbinaticns of modules to teach 
vocabulary concepts should the hypothesis be confirmed. 

The seven instructional modules are described in the report, 
in terms of their instructional routines; i.e., the sets of stimuli. 



responses and feedback each contains. The rationale for each 
module is presented, citing whether the module evolved from an 
instructional design point of view or whether it is an applica¬ 
tion of psychological theory. 

To achieve relatively tight control over student learning 
processes, a CAI (Ccrputer-Assisted Instruction) mode is desirable 
to instrument the research. We have accepted delivery of two 
VT-52 DECSCQPE terminals and two sets of Vadic Corp. modems. 

The first stage of research requires calibrating the size of 
the direct and indirect instructional effects; later stages will 
require studying how the combined use of the methods confirms or 
disconfirms the decision theory equations for optimal combinations 
of modules. In summary, the main accomplishments to date of this 
research endeavor include: 1) A formalization of the decision 
theory framework that guides this research; 2) Descriptions of 
the seven instructional modules and a rationale for each; 3) The 
introduction of a cognitive psychology perspective on the design 
research so as to aid understanding of learning; 4) The detailed 
design of the seven instructional modules plus the pretest; 

5) Development of the instructional database for 24 words for 
six of the modules; 6) Design of the criterion tests; and 7) 
Preliminary testing of the effectiveness of two of the instruc¬ 
tional modules. 

A tryout of the DEFINE and CLASSIFY modules (in adapted 
pencil and paper form) was conducted to determine the existence 
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of indirect effects. All of the instructional routines making 
up each module were used. Twelve students were assigned randomly 
to each of two instructional groups. Comparison of data obtained 
on six of the post-tests shewed evidence of indirect effects. 

When the quality of perfomanoe on three post-tests was compared 
with the quality of perfonrance on the post-test that matched in¬ 
struction it appeared that Define instruction led to greater in¬ 
direct effects than Classify instruction. The differences were 
not statistically significant but the trends suggest different 
sized indirect effects for different modules a;d this will be one 
of the questions relating to optimal carbinations of modules which 
wall be pursued in forthcoming studies. We have also decided to 
assess indirect affects under conditions of instructional treat¬ 
ments with pre-defined proficiency levels rather than under con¬ 
ditions defined in terms of a fixed amount of time or number of 
items. 


3 












TABLE CF CONTENTS 


The Setting and Development Objectives.5 

Theoretical Framework and Research Objectives.5 

The Instructional Modules.11 

Define Module.11 

Classify Module.14 

Discover Module.15 

Wbrd Relations Module.17 

Word in Context.19 

Word Line Module.20 

Equivalents Module.21 

The Computer System.23 

Two Vocabulary Studies.24 

Study I.24 

Study II.26 

Results and Discussion.28 

Indirect Effects.29 

Future Work.33 


4 




















The Setting and Development Objectives 

The CAI development that we wish to report in this paper is 
directed toward enhancing vocabulary knowledge of low-verbal 
skills college students. The instructional design of the CAI 
system is a simple pretest, instruction, posttest design. The 
words to be taught have been selected from existing frequency 
lists with special attention to their "power" (occurrence in a 
wide range of reading materials). The sites at which the in¬ 
struction will be used will be developmental reading laboratories 
at the canpuses of a university and/or a ccrmunity college. 

In this paper, we wish to report the progress we have made 
on the design of our CAI system and the results of two studies 
that have helped us sharpen the methodology of cur instructional 
research in vocabulary and that have provided preliminary infor¬ 
mation on the relative effectiveness of two of the instructional 
•.nodules. 


Theoretical Framework and Research Objectives 

We are developing a variety of strategies for instructing 
the words that, according to the pretest, a given student may 
need to learn. Same of the strategies we have built are similar 
to those found within developmental reading materials; others 
are cruite different (see later section for detailed descrip¬ 
tion of the strategies). We have specified a number or per¬ 
formance objectives as the goals of our vocabulary instruction 
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and each of the strategies is focused cn one of these performance 
objectives. Our objectives (0^) refer to terminal behaviors 
such as being able to define a word, use it in a sentence, 
classify instances of the word properly and so forth. 

There are seme particularly intriguing aspects of our 
instructional situation. Previous research (e.g., Johnson and 
Stratton, 1966) has shown that one of the likely characteristics 
of several of our instructional strategies is that they may pro¬ 
vide more instructional returns than cne might ordinarily expect. 

It is likely that a given strategy will have an effect on the 
dt ective it is directly related to and on other objectives. 

Each of our instructional strategies, therefore, appears to con¬ 
tribute to the acquisition of more than one objective. Thus, 
we might restate the matter by saying a strategy has both a 
direct effect (on the objective it is directly related to) and 
an indirect effect (cn other objectives). The interesting problem 
as we see it is to develop rules for deciding the best mix of 
strategies, when certain levels of achievement on tests of the 
objectives are desired. The rules, of course, would be formulated 
to take into account both the direct and indirect effects of the 
given strategies. The rules will also take into account the 
instructional "costs" associated with given individual modules 
or combinations of modules. We currently view "costs" to be amount 
of student time. 
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In collaboration with Guy Groen, we have developed a quanti¬ 
tative representation of the design problem we face and it takes 
the following form: 

1. There are n learning outoanes, 0^ (objectives). 

2. For each 0^, there exists a method and a criterion 
test, CT^. 

3. yields a higher score on CTV than does on CTj. 
Garment : This assumption reflects the admonitions of 
Gagnd and others that instruction that is highly con¬ 
sistent with tests that measure its effects leads to 
greater achievement than instruction that is less 
consistent. We shall drop the assumption if it does 
not hold in our particular application. This v.ill in 
turn mean that we can allow arbitrary selection of 
M^'s when composing prescriptions. 

4. Each Jb has an associated cost, c^. 

5. Each CI\ has a lower bound of acceptable performance, 

“>r 

6. Each has a direct effect > o on Clh and an 
indirect effect o.. > o on each CTj (j / i) such that 

{S} 

CT i = i a ij 

jeS 

when S is the set of methods being used. 
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Garment i t The 's are supposed to be fixed effects. 

Thus, CIV varies depending on which a^'s 
are in the linear combination. 

Comnent ii ; Equation (6) is to be considered 
locally a ceptable. 

Clearly, as the formulation shows, the central problem for 
this stage of our research is hew to determine the c^'s, since 
they are the determiners of level of achievement on a given 
criterion test. We cure adopting two approaches to determining 
the a^j's. One approach we call bottom-up , which means that we 
shall determine the 's from the pattern of criterion test scores 
that are obtained after students receive instruction via a given 
set of modules. The second approach we call top-down and it 
assumes that each of the modules has a structure that can be de¬ 
noted in various ways. Cne model of structure that we have worked 
out so far involves simply decomposing the modules into their 
instructional routines. Each module can be analyzed into com¬ 
ponent routines serving different psychological functions, and 
some routines are common to sane modules. 

Both approaches, bottom-up and top-down, lead to int" esting 
testable predictions about patterns and levels of achievement 
on the criteria^ tests as a function of what modules are used 
in the students instructional prescription. We shall not explore 
the full set of predictions that can be generated frem the model 








here; we shall, however, show one interesting consequence of (6) 

nn the previous page* It is possible to derive the theorem: 

7 { 1 , 2 } { 1 , 2 } { 1 } { 2 } 

cr 1 + cr 2 > ci^ + cr 2 

The theorem can be shown to be true in this manner: 



The above theorem is tested in an experiment comparing student 
achievement sunned on two criterion tests as a function of in¬ 
structional modules (1 or 2 only vs. 1 and 2) received. Our 
research in the forthcoming months will be directed toward test¬ 
ing tneorems such as (7). We shall initially be concerned with 
developing our prescriptive ules as they apply to groups of 
students; eventually we hope to be able to apply the formulation 
we have developed to the learning cf individuals, for one might 
reasonably expect individuals to differ in their propensity to 
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transfer (i.e., the magnitude of indirect effects ought to vary 
among individuals). 

Exploration of the previously described model K t we think 
governs construction of instructional prescriptions in vocabulary 
shavs that it has seme interesting consequences. The key concept 
of the model is the existence of indirect effects. If indirect 
effects do not exist, then the question of what constitutes the 
most appropriate prescription is uninteresting, because in order 
to develop each one of a set of performance objectives, each of 
the methods must be used. Therefore, the major purpose of the 
first stages of our research has been to demonstrate that in¬ 
direct effects exist. Of course, to enhance the chances that the 
methods we develop do in fact produce indirect effects, we have 
incorporated the methods used in the 1966 study which were, in 
turn, based upon methods found in the developmental reading 
literature. 

Our purpose in this paper is to describe the seven in¬ 
structional modules we have developed in terms of the design 
guidelines used to create them and in terms of their oenponent 
routines. We also include displays of the instruction as it will 
appear on the CRT, for visual displays show instructional 
task content in ways that are not easily captured in text des¬ 
criptions. We shall also describe seme research of a preliminary 
nature that bears on the question of the existence of indirect 
effects. 
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The Instructional Modules 


This section will describe the DEFINE, CLASSIFY, WORD 
RELATIONS, WORDS IN CONTEXT, WORD LINE, DISCOVER and EQUIVALENTS 
modules. Each module carprises smaller segments of instruction 
called routines - each routine corresponding to a change in the 
stimulus-response situation. The routines were designed to follow 
a shaping procedure, i.e., gradually reducing the stimulus support 
for the terminal response as the module precedes. Routines are 
of two kinds: expository and inquisitory. Sane routines occur 
in more than one module. Table 1 lists and briefly describes 
all routines for all seven modules and can be consulted to com¬ 
pare them. It will be noted that seme routines use a response 
contingent criterion for exit; others do not. 

Insert Table 1 about here 


Define Module 

The define module consists of seven routines. 

Routine 1; Pronounce. This routine presents the target 
word and a phonetic pronunciation key, with the direction to the 
student to pronounce the word. This routine begins six of our 
seven modules. 

Routine 2: Definition. An extended definition of the 
target word is shown, emphasizing the critical attributes by 
separating them visually from the rest of the display. The 
student is instructed to read the definition. 
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Modules and Their Component Routi 






Routine 3; Type fran Memory. While the definition remains 
on the screen the student is instructed to look carefully at the 
word until ready to type it frcm memory. The student then types 
the word after the stimulus disappears fran the screen. Feed¬ 
back either confirms a correct response or - in the case of 
an incorrect response - presents the word again, which is re¬ 
typed by the student with the word in view. (Figure 1 presents 
screen displays for DEFINE, Routines 3, 5 and 6). 


Insert Figure 1 about here 

Routine 4: Definition with Attribute Labeling. First the 
form class of the target word is identified along with a general 
statement of what a definition of this form class should con¬ 
sist of. Then a short form of the target word's definition is 
presented with the parts labeled according to the preceding 
statement. 

R outine 5; Multiple Choice Paraphrase. In this routine 
part of a definition paraphrased fran the original definition 
appears on the screen; it is incomplete because one of the 
critical attributes is left blank. The student selects fran 
three choices the paraphrase of the missing critical attribute. 
Feedback for an incorrect response tells the student to pick 
again. This routine is repeated for the number of critical 
attributes contained in the target word. 

In this routine an option is introduced. The definition 
frcm Routine 4 can be either present or absent while the multiple 
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define 


CLASSIFY 


Routine 3: Type word from memory 



Routing 3: Matchaa Exampla and Non-example 



Routing 5: Multiply choica paraphrase of aach of the 
dafinltion'i critical attrlbutai 



Routine 6: Multiple choico peraphrme of all 
critical attribute* 



Routine 4: Forced choica of axampla 



Routine 6. Practice 



Figure 1. Simulated screen display for OEFINE and CLASSIFY. 









choice paraphrasing occurs in Routine 5. This and other options 
in other modules allow for variability in what we believe to be 
important instructional parameters. These options allow us to 
make some modules more similar or less similar to each other and 
to isolate parameters that theoretically might produce signifi¬ 
cant differences in achievement. 

Routine 6: Multiple Choice Definition Construction. In 
this routine the general statement of the form of the definition 
(from Routine 4) is given. The student selects a correct para¬ 
phrase (different from those in Routine 5) for all of the parts 
of the definition. This routine thus begins to approximate 
definition construction. Feedback for an incorrect response 
directs the student to try again. After two incorrect re¬ 
sponses, the correct definition is shewn. 

Routine 7: Definition Construction. In the final routine 
of this module the student is instructed to type a definition 
of the target word. She/he is cued to start by typing the 
target word and to phrase the definition appropriately for a word 
in its form class. After making the response, the student com¬ 
pares his or her definition with a presentation of the original 
definition. 

The criterion test for this module is similar to Routine 7, 
definition construction, minus the feedback. 

For the remaining modules only those routines that are dif¬ 
ferent from the routines in the DEFINE module will be described. 
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Classify Module 


Routine 3: Exanple/Nan-exanple. After presenting the word 
and its pronunciation plus the definition, CLASSIFY next pre¬ 
sents a situational exanple of the target word and a matching ncn- 
example that differs from the example on only one attribute. 
Example and non-example are labeled and a question is asked which 
requires the student to attend to the presence of the attribute 
in the example and its absence in the non-example. The student 
responds by typing either EXAMPLE or NCNEXAMPLE. Feedback for 
an incorrect response calls attention to the attribute the stu¬ 
dent should have attended bo in making the response, then directs 
the student to choose again. This routine repeats for each 
attribute of the target word then a final pair is presented which 
varies all attributes at once. (Figure 1 presents screen dis¬ 
plays for CLASSIFY Routines 3, 4 and 6). 

Routine 4: Forced Choice of Exanple. Next a matched 
exanple/non-exanple pair of situations is presented unlabeled. 

The student is instructed that only one of the instances is an 
exanple of the target vord and is asked to identify it by typing 
its number. Feedback indicates whether the response is correct 
or not and if not, the specific wording in the exanple that 
"gives it away" as the exanple is capitalized. The student 
chooses again. Note that in both Routines 3 and 4 the student 
must make the correct response before receiving the next item. 
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This routine is repeated until a total of six or seven example/ 
non-exartple pairs have been presented in both Routines 3 and 4. 

Routine 5: Forced Choice Example Sumnary. This routine 
re-presents only the examples frcrn the previous routine. They 
are identified as examples and the student is instructed to note 
haw they are alike and hew they differ. 

Option: The short form of the definition can remain on the 
screen during Routines 3, 4 and 5. Having the definition on makes 
this module more similar to DEFINE. 

Routine 6: Practice. In the final routine the student is 
presented with one instance at a time with the direction to type 
the target word if the instance is an example or type "Not (target 
word)" if it is a non-example. This routine is repeated four times 
with the instances being randomly varied between examples and non- 
examples with the constraint that at least one example and non¬ 
example shall appear among the four instances. Feedback for an 
incorrect response labels the given instance as either EXAMPLE 
or NCNEXAMPLE and presents the matching instance. The student 
is directed to read and compare the two instances. This routine 
re-presents missed items after all items have been presented once. 

The criterion test for this module is similar to Routine 6, 
minus the feedback and the re-presenting feature. 

Discover Module 

This is a discovery version of CLASSIFY, the rationale being 
that a task requiring the student to induce a concept from 
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exanp3.es might aid in the development of the critical attributes 
in memory. The routines are largely the same as in CLASSIFY but 
there are two important differences. First, the definition is not 
presented until the end in DISCOVER. Second, in place of 
CLASSIFY'S inquisitory exanple/non-example routine, DISCOVER has 
an expository Introductory Examples routine which siirply presents 
a labeled, matched pair without calling attention to the critical 
attributes in the exanple. 

The DISCOVER module then goes into the Forced Choice of 
Exanple routine, identical to that in CLASSIFY, and then into the 
Practice routine. 

Options; The example/nan-exanple pair fran the Introductory 
Exanple routine and those pairs fran the Farced Choice of Examples 
routine can remain on the screen throughout the latter routine. 

If this option is chosen, then the Practice routine is identical 
to Practice in CLASSIFY (the examples having been erased from the 
screen). If the option is not chosen then Practice takes a slightly 
different form. Instead of presenting only four instances to be 
classified and reserving their matching instances for feedback, 
the matching instances are included in the instance pool, thus 
giving four exanple/nan-exairple pairs, randanly arranged. Feed¬ 
back for an incorrect answer siirply indicates that the answer is 
wrong and directs the student to try again. 
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The final routine presents the short version of the definition 


with attributes labeled. 

The criterion test is the same as for CLASSIFY. 

Word Relations Module 

This module begins with the pronunciation and definition 
routines described earlier. The third routine, Forced paraphrase, 
is similar to the Multiple Choice Paraphrase routine in DEFINE 
except that instead of paraphrasing each separate attribute a 
paraphrase of the target word is selected. (Figure 2 presents 
screen displays for WORD RELATIONS Routines 3, 5 and 6). 

Insert Figure 2 about here 

The characteristic routines for this module begin with 
Routine 5, Explanation of Relations. This routine illustrates 
hew two words can be related to each other by presenting two 
examples of a particular relation. The first exanple is an arche¬ 
type of the relation, e.g., "Helpfulness is a QUALITY of a Boy 
Scout." The second example illustrates the same relation for 
the target word, e.g., "Insincerity is a QUALITY of a sycophant." 
The student is then directed to select another word that bears 
the same relation to the target word. This routine is repeated 
for three or four different relations. Feedback identifies the 
response as correct or incorrect and if incorrect, the student 
is directed to try again. 
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WORD RELATIONS 


WORDS IN CONTEXT 


Routine 4: Story end Queitiont 


One of these phrases captures the meening of 
SYCOPHANT. 

1) e herd-working 21 e fett-telking 31 e weelthy 
echiever apple-polifher fat perton 

Type the number^ 


For example. 

if helpfulneji - it a QUALITY - of a Boy Scout, 
then insincerity - ii a QUALITY - of a sycophant 


I**??**) could also be a QUALITY of a sycophant. 
1) honesty 21 kindness 31 ambition 4) fairness 


King Foolish liked to be surrounded by members 
of his court. He spent his whole day listening to 
them flatter him to build up hit ago. 

One of the attendants, who thought he might be 
rewarded by the king with a purse full of gc'd, w*s 
full of praise. This attendant was always saying, 

"King Foolish, how wonderful you erel King Foolish, 
you're the wisest ruler a kingdom could haval King 
Foolish, you're the greatest!" 

Another attendant wanted to be made a knight. 
This attendant had a slightly different approach to 
winning the king's fevor. Ha was constantly kneeling 
before the king end kissing his hand. Whan the king 
was around, this attendant never got off his knees. 

The queen watched what was going on among the 
members of the court. "Humphl" she snorted one 
day when she was fed up. "They're sycophants, ell of 
them. Each of those attendants is about as sincere as 


The queen called the attendants sycophants because: 
11 they buttered up the king to get favors. 

21 they flattered the king to meke her jealous. 


3) they praised the king to make h 


How Is charm related to a sycophant? 

Ills a JOB of 2) is a TOOL of 3) is a GOAL of 


Routine 8: Sentence Compl 


Complete this sentence in your own words. 


A student might be called a sycophant if he 


Press return when you have finished. 


Compere your answer with this sample answi 
A student might be called a sycophant if he 
flattered the teacher in order to get a high 
grade. 


Figure 2. Simulated screen display for WORD RELATIONS and WORDS IN CONTEXT. 
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Opt 4 "m; The sho r t form of the definition can remain on the 
screen during Routines 3 and 5. 

In Routine 6 a word is presented aJong with the target word 
and the student selects the name of the relation between the two 
words. This routine continues through several examples for each 
of several relations, different ones from those used in Routine 
5. In writing instructional materials for this module we work 
from a list of 15 possible relations. The list includes, for 
example: CAUSE of, RESULT of, TOOL of, TARGET of, LOCATION of, 
PART of, etc. Not all the relations are applicable to each 
target word, of course, but for those that apply four or five ex¬ 
amples of each relation are written. This routine repeats for 
about 20 questions, presenting 2 to 4 questions for each appro¬ 
priate relation. Feedback for an incorrect response identifies 
the answer as wrong and directs the student to try again. Missed 
items are re-presented at the end of the series. 

The final routine, Relation Summary, lists all the relations 
taught for the particular target word and re-presents an example 
of each relation. The student is required to type the target 
word several times in this routine. 

The criterion test is similar to Routine 6, minus feedback. 
Because this test is based upon a model of memory, we consider 
it to be experimental in nature. It will be of great interest 
bo us to determine whether a subject receiving instruction solely 
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via WORD RELATIONS will reflect a difference in cognitive structure 
frcm those subjects receiving instruction via the other modules. 

Word in Context 

The characteristic routines in this module present a story 
of about 160 words and a series of six or o^ven questions which 
relate details of the story to attributes of the target word 
(see Figure 2). The target v*ord appears in the story only once, 
toward the end. The stories are often generated from one of the 
examples from CLASSIFY. As an aid in generating the questions 
Stein and Glenn's (1977) schema for analyzing story carprehension 
(derived from Rumelhart) served as a guide in selecting those 
units of a story that are most likely to be remembered. 

The questions vary in format: multiple choice, yes/no and 
short answer. Feedback for an incorrect response identifies the 
answer as wrong and directs the student to try again. 

Option: The story can remain on the screen while the ques¬ 
tions are presented. If this option is chosen, feedback for an 
incorrect response also includes underlining the pertinent words 
of the story. 

The last question presented is in the form of a multiple 
choice sentence aorrpleticn. This approximates the terminal task 
of the module. 

The final routine requires the student to finish a sen¬ 
tence, the given part of which includes the target word. The 
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student response calls for an explanation of the word. For exanple, 
"A student might be called a sycophant if he . . 

The criterion test for this module is sentence completion. 

W ord Line Module 

This module is identical to WORD RELATIONS for the first 
four routines. The first new routine for this module, Routine 
5 Illustrative Line, introduces the student to a word line, 
demonstrates how it is used and gives the student brief practice 
on the task that will follow. After having done this routine 
once, the student has the option of skipping it the next time 
he/she works through this module. (Figure 3 presents screen 
displays for WORD LINE Routines 5, 6, 7 and 8). 

Insert Figure 3 about here 

In Routine 6, Map Line, the word line presents the target 
word in a group with five other words of the same form class and 
requires the student to arrange them along the line according to 
some semantic dimension (usually the major attribute of the target 
word drawn frcm the definition). In Figure 3, for exanple, the 
dimension of the word line is: to make larger/to make smaller, 
for the target word "augment." The other words in the group in¬ 
clude one synonym of the target word, two antonyms and two words 
that fall into the middle ground along the given dimension. Note 
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that the instructions avoid irrplying that the word line repre¬ 
sents a continuum with subtle gradations. The task essentially 
requires the students to sort six words into three categories; 
in this case, the categories are to make smaller, to make larger 
and to do something that neither makes smaller nor larger. Two 
words go into each category and the position of the words within 
categories is not important. Feedback identifies the response 
as correct or incorrect and places the word on the appropriate 
section of the line. 

In Routine 7, Rate Line, the student is given a short word 
line with the target word and one other word in each category 
in position on the line. Six instances are presented and the 
student must select their appropriate positions on the line, thus 
relating an instance to a word on the line. 

In Routine 8, the final routine, questions are asked and 
the student responds with the words fran the word line. This 
provides two opportunities for the student to write the target 
word (plus the earlier opportunity in the Type fran Memory 
routine). The criterion test for WORD LINE is similar to Routine 
6, the student arranges the target word and five other words on 
a line. 


livalents Module 


The Equivalents module is a very simple comparison module. 


It consists of a single routine which presents repeated practice 


21 






on items missed on the multiple choice pretest. The format is 
the same as on the pretest except for feedback. Confirmation 
is given following a correct response; in case of a wrong response, 
the correct answer is identified. Items continue to be re¬ 
presented until the student has responded correctly a given number 
of consecutive times. The option for this module is that the num¬ 
ber of consecutive correct answers required may vary. An important 
feature of this module is the use of an item presentation strategy 
that attempts to maintain a lag of three interpolated items between 
a word's n* and (n + l) 2 ^ presentation. 

Insert Figure 4 about here 

Figure 4 illustrates the computer management of the instruc¬ 
tional system. The left column (Input) refers to the various 
computer programs that process the instructional material for 
presentation to the student on the DECscope. The output pro¬ 
gram collects and prints out the student response data. The 
Student Use Package refers to the programs that test the student 
and present the words missed on the pretest for instruction via 
one or more modules. 

There will eventually be two modes of system use; Instruc¬ 
tional and Experimental. At present we are in the Experimental 
mode. In this mode the experimenter specifies how many words 
the student will learn and which module or modules will be 
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Figure 4. CAI Vocabulary Instructional System 













needed for instruction. Later, in the instructional mode, 
prescriptions specifying combinations of modules will be developed 
from the experimental findings and tested for optimum achievement. 

The Ocnputer System 

The vocabulary instruction is coded in Fortran IV and will 
be delivered via Digital Model VT-52 Video Display terminals with 
a 120 character/sec. transmission rate, connected to a PEP-15. 

The time-sharing system, ETSS, was developed at the Learning Re¬ 
search and Development Center (Fitzhugh, 1973). The system con¬ 
sists of a DEC PDP-15 computer, over 100 megabytes of high speed 
disk memory, and a terminal controller capable of supporting 
up to 32 work sites. In addition, a small DEC PDP-11 ccnputer is 
attached to the PDP-15 and controls a high speed line printer, 
a card reader and a card punch. An on-line text editor, a com¬ 
prehensive statistical package and a variety of utility systems 
are also provided. Several applications programsrs develop the 
instructional programs according to the specifications of the 
instructional designers. 

Since our CAI programs are not yet caipleted, the studies 
we report were conducted with paper and pencil materials. In 
the second study, we attempted to achieve as much mament-to-manent 
control over student learning processes as was possible within the 
constraints of a relatively inflexible paper and pencil media. 






Two Vocabulary Studies 

Two studies were conducted in pursuit of determining the 
instructional effects of several modules. 

Study I 

The first study (Rosner, 1978) used materials and a pro¬ 
cedure directly modeled upon the 1966 study noted previously 
that served as our initial empirical justification for the exis¬ 
tence of indirect effects from vocabulary insturctional strate¬ 
gies. The purpose of Posner's study was in part to determine 
whether the sizeable indirect effects found with the use of a 
definitions method could be replicated in our laboratory. She 
taught four difficult vocabulary words that were identified on 
the basis of a pilot study and word frequency counts to college 
students. Unfortunately, despite the fact that the difference 
between the number of words her instructional groups mastered 
ccnpared to her control group was comparable to the difference 
between instruction and control groups in the 1966 study, her 
difference was not significant. Thus, it was not meaningful to 
discuss the effects obtained via the instruction. 

We suspect that most of the difficulties of the study were 
procedural. First, students in the 1966 study kept the materials 
for 12 minutes; Rosner's students paced themselves for an average 
of 7 minutes with the material. Recent research on learning 
within CM or OH environments has shown that problems students 






encounter center around deficiencies in self-management skills 
(see Judd, McCombs & Dobrovolny, 1978) and they need skill 
management training. It seems likely, therefore, that students 
sirtply did not manage their word study effectively. A second 
problem is concerned with defining an appropriate degree of con¬ 
trol over student learning processes when only a single frame is 
devoted to a particular word (as was the case in the 1966 study 
for the definitions method). This level of instructional control 
over student learning processes is fairly gross; students can 
make the single required response for each instructional word, 
then attention can wander for the rest of the interval (in sane 
students) or appropriate study behavior will take place (in 
others). Perhaps it is the case that the naturally occurring 
distribution of learner processes during a "free" period is the 
significant determiner of achievement and not instructional 
method (at least as such methods were implemented in the 1966 
study). Since Posner's learners spent a briefer time in instruc¬ 
tion than those in the 1966 study, it is possible that had more 
time been required (or multiple frames per word been used) achieve¬ 
ment might have been increased. The lack of significant differ¬ 
ences may have been due to a higher level of prior learning of 
the instructional words than was expected. Finally, it is carmonly 
found that undergraduate students are not often motivated to per¬ 
form well in psychological experiments. We expect that the 
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responsive instruction provided by our CAI will induce more 
substantial levels of learning. 

We improved the methodology of the second study by using 
multiple frames to instruct a word and the frames were modeled 
after two of our CAI modules. This change had the effect of 
increasing instructional time and theoretically, achieved more pre¬ 
cise control over student learning processes within a given in¬ 
structional interval. We also used a pretest as a method for 
identifying previously unknown instructional words. Thus, our 
second study is modeled as closely after our instructional system 
design as we could accomplish within the constraints of paper 
and pencil materials. A second constraint was that we had to use 
a fixed number of instructional sessions because our subjects 
were recruited from a small educational psychology class. 


Purpose. The major purpose of the study was simply to 
verify the existence of indirect effects under the instructional 


conditions we had designed. The modules tested were those for 
which material had been written and tested, the DEFINE module 


and the CIASSIFY module (in adapted paper and pencil form). In 
the paper and pencil adaptation, we used all of the instructional 
routines that defined each module, adapted as best we could to 
a paper and pencil programed instructional format. The 


26 








instructional materials consisted of a series of frames for each 
word that followed a shaping procedure. 

Design and Procedure. Ten infrequent words were taught to 
two groups of students recruited from an introductory educational 
psychology class. There were 12 students randomly assigned to 
the two instructional groups. The materials for each instructional 
word consisted of five pages of instructional frames with response 
feedback typically at the top of a "next" page. Students also 
had to use a paper cover (laminated plastic sheet) designed to be 
used to cover up certain portions of the previous text material. 

The paper cover was our attenpt to mimic the erasure capability 
of a CRT. The words were presented in a different randan order 
to the students. 

In the first session, students took a multiple choice pre¬ 
test requiring identification of a verbal phrase equivalent in 
meaning, learned the ten words (some of which they already knew 
as indicated by the pretest) and then took an immediate definitions 
post-test. It was not possible given constraints under which the 
study operated to teach students only the words they missed. 
Students returned exactly one week later for retention tests 
(they were not informed they would be tested). They were given 
six tests - free recall, definition-stating, sentence generation, 
sentence understanding, classification; the multiple choice pre¬ 
test also served as a post-test. The free recall test was 
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included because we believe modules other than the ones used in 
the present study will influence free recall, and we wanted a 
measure fran the two methods of interest in this study. The tests 
were ordered as noted to minimize learning fran one test to the 


Results and Discussion 

Time and total learning. There was a significant difference 
in the mean amount of time spent learning in the two groups. 

Mean time spent in Define instruction was slightly longer (X ■ 

29.4 minutes) than in Classify instruction (X = 25.7 minutes) , 
a difference of 3.7 minutes (t = 2.40, df = 22, p < .05). We do 
not, however, think the time difference is large enough to be 
practically significant or to strongly bias our achievement 
results. Our mean time per word was close to the 1966 study 
which used 12 minutes for four words. 

The two groups did not differ in prior level of word 
knowledge when entering the instructional treatments; the Define 
group knew an average of 6.7 words, while the Classify grcup knew 
an average of 5.8 words (t = 1.25, df = 22, p > .05). A t test 
of the mean number of words gained in each group using scores an 
the multiple choice test (a test of general vocabula ry knowledge) 
that was given as a pre and post-test shewed both groups gained 
an equivalent amount (X gain for Define was 2.67; X gain for 
Classify was 3.10; t = 2.074, df = 22, p > .05). The Define group 
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obtained 94% (6.7 + 2.7) overall, while the Classify group ob¬ 
tained 79% (5.8 + 2.1). At test of the scores earned on the 
multiple choice post-test showed the means were not signifi¬ 
cantly different (t = .921, df - 22, p > .05). 

A caparison of sac res on the imnediate definitions test 
with the delayed definitions test (see scoring system, p. 30) 
revealed higher average imnediate scores for both groups and no 
significant difference between the two groups in proportion of 
new words learned, then forgotten (t = .549, df = 22, p > .05). 
There were no differences in the number of instructional words 
that were recalled in the free recall test (X for Define was 
4.0; X for Classify was 3.9; t = .11, df * 22). These data are 
decidedly consistent with the proposition that in terms of total 
amount of achievement, type of instructional method does not 
make a difference. 

Indirect Effects 

The concepts of direct and indirect effects require that 
we consider instructional methods not cnly in terms of contri¬ 
bution to an outcome of general interest but in terms of patterns 
of achievement across multiple specific objectives that might be 
desired as outcomes of an instructional program. 


Insert Tables 2 and 3 about here 
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Tables 2 and 3 shew the mean performance on several criterion 
tests of the instructional objectives as a function of instructional 
method (the sentence understanding test is not included as we have 
not arrived s'- a reliable scoring system yet). The data are pre¬ 
sented for all instructional words (Table 2) and for only those 
words learned in the course of the instruction (the "new" words 
[Table 3]). 

The total number of points that could be earned varied from 
one criterion test to another. The definitions post-tests was 
scored 0, 1, 2 depending on number of attributes mentioned in a 
definition (reliability = .90; maximum points = 12x10x2 = 240); 
the classification test ccrprised 8 instances per word (maximum 
points = 12x10x8 = 960); the sentence generation test was scored 
0, 1, 2, 3 depending on meaning and usage (reliability = .90? 
maximum points = 12x10x3 = 360). The data for all words in 
the Define group are clearly comparable to the 1966 study; there 
are, however, some discrepancies in the Classify group. The 
discrepancy in the sentencing test is easily explained; our test 
required complete sentence construction (using the word "because") 
whereas tht 1966 study required sentence c Tpleticn (a meaningful 
stem was given). There were procedural differences in concept 
teaching methods between the 1966 study and ours which is probably 
responsible for our classify students lower defining perfon ice. 

The data for new words only show lowered performance on the defining 
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and sentencing tests as would be expected for words that students 
have no prior knowledge of. Hie pattern of levels of achievement 
shown for all words is reflected in the new words only for the 
defining and sentence generating tests. The classification test 
retains a high level of performance because it is an easier test. 

A test of the largest achievement difference (Define vs. Classify 
for all words) revealed no significant differences: t = 2.006, 
df - 22, p > .05. We are tempted to conclude, as was done in the 
1966 study, that transfer fran one instructional method to tests 
based on other methods is 100% and therefore we have evidence of 
substantial indirect effects. 

Within a CAI context, however, it is standard procedure to 
define criteria that individuals must meet before they can exit 
a particular segment of instruction. Both studies, ours and the 
1966 study, fixed the amount of time and material to be covered 
within an instructional treatment, and did not set a mastery 
criterion. Therefore, in order to studv the indirect effects 
that may be associated with a particular instructional strategy, 
we must assess those effects contingent upon a performance standard. 

Insert Tables 4 and 5 about here 

Tables 4 and 5 show the quality of performance on the several 
criterion tests as a function of the quality of performance on the 
criterion test matched to the instructional method. In each 
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TABLE 4 

Quality of Performance on the Classification and Sentence 
Generation Posttests as a Function of Quality of 
Performance on the Definitions Posttest for 
the DEFINE Instructional Group. 


Type of Performance 
on Define Post 


Number of 


Subj ects 

Frequency 

Good 8 

18 

Poor 5 

14 


Type of Performance 


on Other Posts 

Sentence 

Classify _ Generation 


Good 

P 

Good 

P 

18 

1.0 

16 

.89 

8 

.57 

3 

.21 







TABLE 5 


Quality of Performance on the Definitions and Sentence 
Generation Pasttests as a Function of Quality of 
Performance on the Classify Posttest for the 
CLASSIFY Instructional Group. 


Type of Performance Type of Performance 
on Classify Post on Other Posts 


Define 



Number of 
Subjects 

Frequency 

Good 

p 

Good 

12 

38 

10 

.27 

Poor 

7 

12 

2 

.16 


Sentence 

Generation 

Good p 


16 .42 


2 .16 









instructional group for each new word and each student, the 
occasions cn which the post-test score was excellent (a score of 
2 for the definitions pose; a score of 7 or 8 instances classi¬ 
fied oorrectly out of 8 instances) and the occasions on which 
it was poor (a score of 0 for the definitions post; a score of 
less than 7 instances classified oorrectly) were noted. For 
these two classes, observations of the quality of performance of 
the same student cn the same word were made on the other post¬ 
tests. For exanple, in Table 4, of the 18 observations of high 
quality definitions, there were 18 observations of high quality 
classification performance and 16 observations of high quality 
(a score of 3) sentence generation. It can be easily seen from 
a comparison of Tables 4 and 5 that the consistency in the 
quality of performance given cn a defining post (Table 4) is quite 
different than that given an a classifying post (Table 5). The 
trends evident in the tables tend to show that Define instruc¬ 
tion leads to irore direct effects than Classify instruction, 
for high quality performances tend to be maintained to a greater 
degree. However, a test of the significance of the difference 

in the proportions using the normal approximation to the Irwin- 

2 

Fisher exact test shewed no significant differences (X =2.39, 
df = 3, p > .05). While the construction of prescriptions is 
more interesting when methods produce differently sized indirect 
effects, our preliminary data have suggested the indirect effects 
may be similar rather than different in magnitude. 
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We have, therefore, decided that within a CAI context it is 
more appropriate to assess indirect effects under conditions of 
instructional treatments with pre-defined .proficiency levels 
rather than conditions defined in terms of a fixed amount of 
time or number of items. This does not negate the formulation 
presented previously. What we have done is to clarify the defi¬ 
nition of indirect effects in a criterion referenced (rather 
than norm referenced) context. The design of our CAI system 
reflects this clarification: We have incorporated an interim 
test for each module whose format is matched to the module. Stu¬ 
dents cannot exit from a given module until the interim test is 
passed. 

Future Work 

In the near future, our research will concentrate on testing 
for indirect effects and testing predictions fran our model, such 
as theorem number 7 mentioned previously. Our goal is to develop 
the prescriptive rules that allow us, through suitable selection 
of instructional strategies, to optimize a given performance 
function across a variety of criterion tests, within certain 
time constraints. As we perform our research we will, of course, 
be providing services to developmental reading laboratories and 
we are anxious to begin implementation at our planned sites. 
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